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ABSTRACT 

Issues in ensuring educational quality are examined 
in this document. Section 1 provides information for developing plans 
and targets. The following areas are highlighted: 

school-effectiveness research findings; developments in Scotland to 
improved educational quality; internal and external evaluation of 
schools; evaluation criteria; the role of the school development 
plan; and strategies for school self-evaluation. The second section 
examines performance indicators, with a focus on the role, 
characteristics, and types of performance indicators; agencies and 
organizations involved in developing performance indicators; the 
indicators produced by Her Majesty's Inspectors; the critical 
performance indicators; and using performance indicators to best 
effect. (Contains 16 references.) (LMI) 
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INTRODUCTION 



The idea for this document arose out of comments offered by Headteachers and others in the 
course of discussions at a number of Audit Unit meetings. It was suggested to us that we 
should pull together, in one document, the issues covered in the keynote talks and discussion 
. groups. 

This we have done. We hope that the document will be of use in school development planning, 
in-service training and as a reference source for meetings and seminars on quality assurance. 
We feel that the paper provides a useful complement to the report on "Quality Assurance m 
Education: Current Debates".* 



HM Inspectors of Schools, Audit Unit 
May 1993 



* Edited by Ian Stronach and published by University of Stirling, 1993 
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SECTION 1 



1.0 DO SCHOOLS MAKE A DIFFERENCE ? 



LI The overwhelming message of HMI and research findings is that schools do indeed 
make a significant difference to the attainment of their pupils. 

1.2 This apparently common-sense conclusion had been questioned in the mid-1970s. 
Research at that time argued that because parental background and other social factors 
were the most important in determining attainment, the contribution of schools could 
be regarded as relatively small and unimportant. 

1.3 It is now recognised that individual schools differ widely in terms of the progress made 
by children with similar backgrounds. Thus, according to Mortimore: 

la general, &e studies locate that - even whm dififerenees : iii intake 
iteclfeife than 
othei^'tolcadtogoodomcomes. ■ 



1.4 This is an encouraging message for all involved in education. It stresses that schools 
can have a significant effect even where there is considerable disadvantage and 
deprivation, for as pointed out by Reynolds: 



wider systems of inequality tod social stratification that affect them. 
Schools 

people*^ * 1 1 

l S ,U_, _ .-l-.f— ,, - , ■ .... ...... ..... , , ; 



positive effects i 
nane effective. 



1.5 The possible magnitude of school effects was illustrated in a Scottish study by Willms 
and Cuttance. After controlling for family background characteristics and pupil ability 
prior to entering secondary school, they found: 



•schoolsis- suggested by the finding that ill 
the;average pupil left with 'five -or more *0 ( Grade passes (A~C> whilst 
in the worst -schools a sMar pupil left wi& csly one or two X)' Grade 
passes. 





1.6 Researchers have identified a number of different sets of features linked to school 
effectiveness and these sets have much in common with each other. Hopkins identifies 
two issues in particular on which the school effectiveness literature is in agreement: 

First, that differences in outcomes are systematically related to 
filiations in the school's climate, culture, or ;^^ : tS^pnd, that the 
;S^6l^ culture is ^^0^t-to- alteration by con<£S;^ 
part of the school :s^£K 
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1.7 



Jf ^f^S <5wiai^ii^ places; -^be^-^M^I^^MW 
ensurc hl ^ h standards of waik ^^havK«m at the same time f! 
^responsive to pupa,, for the teachers ate aPlal 

iiP- ^-^ •wl^HS^WftAtion -of : ure^im a*** which 



2.0 WHY THE CONCERN WITH ISSUES OF QUALITY IN EDUCATION ? 



2.1 
2.2 
2.3 

2.4 
2.5 



^eESES reC ° Sniti0n ° f ^ ° f 311 Pupils in 0Ur «*«* » receive a high 
JSWo&EES^ ° f 6dUCati0n SyStCm CdtiCally «** * e —mic 



3.0 



3.1 



EDUCATION f E D ° ING T ° FURTHER IMPR0VE T HE QUALITY OF 

wS°h n h- < |f vel< f nen ? in re <*nt years have focused on building a framework within 
which high quality education can be provided. They have poinfed toT^quaUty 

* S FmT° te ? m P. rovement a s the central thrust of schools' planning- 

to improve institutional approaches to achieving what hai been described a < 
systemic quality, that is, quality that pervades the instiKTSSSSftoS 
Within £ Cmbr n mg ' md more powerful than ' inoividual quality 
Sample ° Strat6gy ' ^ ^ 3 nUmbCr ° f relatedVelopments on, for 



the curriculum 

32 I™,- 14 P r °g ramm e is a major curricular initiative clarifying learning outcome 
^^^5S2L2? aSS ° Ciated Pr ° Sramme ° f nati0nal '^"/focuses 
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3.3 



b^liS^ has resulted in increased numbers of pupils gaining 

qualifications. It has also promoted a wider range of approaches to teach ing and 
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learning including, for example, the planned development of practical and problem 
solving skills. 

3.4 The 16-18 Action Plan greatly extended the range of appropriate courses in the upper 
stages of the secondary school. 

3.5 Following the Howie Report consultation period, further developments relating to the 
upper stages of secondary education are currendy under consideration. 



the quality of teaching 

3.6 Planned activity and in-service training facilitates effective staff development relevant 
to local or national priorities. 

3.7 Certain training courses receive national support and resourcing. 

3.8 The present introduction of a national scheme of staff development and appraisal 
provides the basis for ensuring that the teaching force can improve their skills and 
acquire necessary new ones, while balancing the claims of individual career 
development with the needs of the school. 

3.9 Revised guidelines for teacher training courses set out the range of competences that a 
new teacher is expected to possess at the beginning of his or her career. 

3.10 There is a growing focus on the support given to probationer teachers. 

3.11 A series of HMI reports is in process of publication giving advice to teachers on 
effective teaching and learning in a range of secondary school subjects and on other 
aspects such as provision for children with special educational needs. Earlier HMI 
reports on effective secondary schools and effective primary schools gave more general 
advice on teaching and learning strategies. 



resources 

3.12 Developments in educational technology increasingly enhance the educational process. 
Learning is facilitated through the effective and efficient use of new media, methods 
and materials. The Technical and Vocational Education Initiative and the national 
resourcing of developments such as Standard Grade are key promoters of this. 



parental involvement 

3.13 The introduction of School Boards has fostered the development of positive contacts 
between schools and parents and heightened awareness of all around in the education 
offered. 



training of headteachers 

3.14 The significant increase in management training for headteachers promotes 
improvements in their leadership qualities, professional competence and capacity to 
develop effective woridng relationships. 



3.15 All of the developments described above are intended to bring about changes in what 
happens in classrooms to the benefit of pupils. The developments have been promoted 
through complementary initiatives in a number of education authorities, often focused 
on valuable pilot work in particular schools. Several education authorities have begun 
to set out their aims in frameworks which provide a context for development work 
across all aspects of educational provision. 



4.0 HOW DO WE EVALUATE THE QUALITY OF EDUCATION ? 



4.1 There must be means of judging whether a quality education is actually being 
delivered; quality must be assured. One way of judging quality is to examine the 
outcomes of the educational process in the school. 

4.2 Academic attainment provides one outcome measure but the attitudes which young 
people acquire and the way in which they develop as balanced and rounded citizens are 
no less important. 

4.3 Judgements on quality should also be made in terms of the processes and conditions 
within the school: whether what the content of what is taught and the learning 
environment created by the school promote desirable outcomes. 

4 4 In education, outcomes are the product of processes occurring over a period of years 
but information is required on an ongoing basis. Judgements about overall quality 
should be based on a balanced consideration of both processes and outcomes. 



5.0 INTERNAL OR EXTERNAL EVALUATION OF SCHOOLS ? 

5 1 There has always been recognition that it is important to know how well schools are 
performing, and a wide-ranging literature on the effectiveness of schools produced 
over the last 20 or so years has established a consensus about common features of 
effective schools. 

5 2 In recent years, the focus of this literature has begun to shift from setting out such 
characteristics towards looking at how a school can develop and bring about 
improvement in ways that enable it to be more effective in achieving its educational 
goals. 

5.3 This concern for school improvement has been highlighted by the emphasis within The 
Parents' Charter in Scotland on standards and quality in education. 

5.4 While research evidence indicates broad agreement on the characteristics of effective 
schools it may be less clear what action a school should take to establish these 
characteristics. The first step on the road to improvement is for a school to gain a clear 
understanding of its current stage of development. 

5 5 Evaluation of education in a school might be undertaken internally by the staff of the 
school Those who engage in self-evaluation start from a position of strength in 
that they are informed insiders with a close and detailed knowledge of the inner 
workings of their organisation. As personal stakeholders they should readily 
accept and act upon their findings. The process of being consulted, having one s 
opinions considered, feeling that one's contributions are seriously valued and that they 
may result in change for the better are a!i powerful builders of morale, confidence and 
commitment. 
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5.6 As Turner and Clift point out: 

UPliliPilve means of improving the quality of toe eduf 
provided by schools is to;gll|ili^^ 
owa.pwfcpliM 
in conscience"/ 

5.7 Alternatively or additionally, evaluation of the education provided by a school can be 
undertaken by the responsible authority. External prompting, guidance and support can 
be important in assisting a school to make judgements on the quality of the education 
provided. Indeed in the absence of such factors a school's own evaluation might simply 
reinforce its existing impression of its situation or avoid the consideration, of difficult 
areas. 

5.8 The Parents' Charter restates the case for independent inspection of schools' 
performance. A system of independent national inspection addresses national concerns 
for standards and quality in education through HMI evaluation and reporting on the 
strengths and weaknesses of provision. 

5.9 Following a revision of HMI inspection procedures, an inspection is a more focused 
agent of change than in the past. Schools and the authority have to produce a paper 
setting out how they plan to address the recommendations in the report and there is a 
further inspection about one year later to assess and confirm progress. 



60 WHAT SHOULD BE THE REFERENCE POINT FOR INTERNAL 
EVALUATION ? 

6.1 The starting point is a recognition on the part of all concerned that they must be aware 
of the quality of what they do and the need for improvement. 

6.2 And building on that, schools need a structure, the parts of which are: 

* clear set of aims, targets and standards to be achieved; 

* mechanisms for delivering them, some of which may exist, other of which 
may require development; and 

* ways of assessing performance against expectation/aspiration and 
communicating the result. 



7.0 WHAT IS THE ROLE OF THE SCHOOL DEVELOPMENT PLAN ? 

7.1 A school development plan is a way of organising relevant information about a 
school's current position and its intentions for the future. The consultation paper 
"School Development Plans in Scotland" issued by SOED is a good reference point. 
The plan provides the means of: 

* articulating and publicising the school's aims and aspirations, and the 
specific targets to be addressed in the s"iort and long term; 

* expressing priorities for development and when these will be achieved; 

* offering evaluation (audit) of performance against the first two. 
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What are the strengths of school development planning ? 

12 There are three main functions of development plans: 

* managing change; 

* sharing of responsibility; and 

* rendering account. 

73 The critical function of the school development plan is to help bring about effective 
action to assure and improve the quality of education provided by the school. 

7.4 The process of development planning enables the school to focus effort on a 
manageable number of agreed priority targets and to judge the degree of success in 
overtaking them. 

7.5 It also encourages the school to maintain commitment to areas which are of continuing 
relevance. Within this structure, school audit is used to identify appropriate priority 
targets. 

7.6 A school development plan provides a focus for the school and the education authority 
to share in the responsibility for assuring the quality of education. 
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8.0 HOW DOES A SCHOOL EVALUATE (AUDIT) ITS PERFORMANCE ? 

8.1 Undertaking a school audit involves school managers first of all in making decisions 
about the aspects of the life and work of the school which are to be audited and how 
they are to judge the quality of the school in each aspect. 

8.2 Education is a complex process. It can be difficult to make precise judgements about 
what constitutes good quality in any aspect. Further, the various aspects of the 
educational process interact with one another. Ail of this means that school audit has to 
be structured carefully. 

8.3 r rhe current focus is on the school because it comprises an administrative unit which 
can be evaluated. However there are variations within schools and within stages or 
departments.. The HMI Audit Unit report Standards and Quality in Scottish Education 
1991-92 shows that there are aspects of every school where improvement is possible. 
For example, in some schools, the quality of learning and teaching within the 
classroom can be improved. In others better planning at a primary stage or secondary 
department may be needed. 

8.4 Responsibility for quality assurance rests at each of these levels. The school will not be 
able to respond to ever-changing demands unless all engaged in its work play their full 
part ir assuring quality. 

8.5 HMFs recommended framework of key areas for audit in Scottish schools is: 



Pnroary School 
.Class/Stage 


Wfeote School 


t. Quality of the ciimcotaro 

lisp 


V Staff i 

il Cumcutum ptemmg 


.' :>:>>.: - :- ;. : ; : ;;.; : ; : . ^ y v:'^ : ■ :■•>?:':' :7 : ':IY: : '':' : : :: :. '. Iv'Yv . . 




... J: - ;-:. -.'; :;..';.". ' . : :. .;::':*;.:.;.':;:'. : :.: 
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Secondary School 
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• . j • •• :;. • ; ' .. • •' • - • ": : ■■ • • • : .' • 


• - / V : --:----v : :: : *..:. •• . .V : j / : ■ 
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: H mi caching 
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pupils, parents ami teachers, and 
disciuttoe) 


* Staff 
• • * Curriculum pi&aniag 
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8.6 Audit within any one of these key areas involves consideration of a wide range of 
factors. Accordingly the frameworks have to be developed further in a way that is 
systematic, manageable and illuminating. HMI recommend that this development 
shculd be based on the use of performance indicators. These are discussed in detail in 
the second part of this paper. Performance indicators provide the means for structuring 
the audit, for selection of areas for focus within it and for facilitating judgements about 
the performance of the school. The next section asks: "Why Performance Indicators?". 
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SECTION 2 



9.0 WHAT SHOULD PERFORMANCE INDICATORS DO ? 

9.1 Education performance indicators allow evaluative statements to be made about aspects 
of education which are large enough to avoid fragmentation but small enough to be 
observed, considered and evaluated separately. 

9.2 In looking at the performance of the education system a method is needed to define 
aims, determine the extent to which these aims are being realised and draw sensible 
conclusions about the next steps which should be taken. In particular, teachers and 
headteachers must start by achieving common understanding in the school of what is to 
be attempted and what would constitute success. 

9.3 Such an understanding may be approached through defining performance indicators 
which: 

* identify key aspects of the school's activities; 

* set out criteria which assist judgements to be made about quality ; 

* point to areas which require detailed and specific investigation ("signposts"); 

* provide a basis for measuring the performance of the school against a set of 
defined criteria ("touchstones"); and 

* enable school managers to make useful decisions about strengths and 
weaknesses. 



10.0 WHAT FORMS DO PERFORMANCE INDICATORS TAKE ? 

Indicators are frequently divided for ease of reference into two broad categories: 



10.1 Quantitative indicators relate to variables which can readily be expressed in numerical 
form. Some examples are: 

* % ofJS5 pupils who gain at least 3 H Grade passes at band C or better 

* % attendance of P7 pupils 

* average cost per pupil 

* pupil-teacher ratio 

* proportion of S4 pupils placed on work experience 

* proportion of parents who attend on consultation evenings 

10 2 Qualitative indicators relate to other aspects in which performance can vary but the 
variables cannot easily be expressed in numerical form; they must be observed, 
described and evaluated. Examples include: 

* quality of pupils' learning 

* quality of the teaching process 

* teacher-pupil relationships 

* meeting pupils' needs 

* partnership with School Board 
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103 Alternative categorisations of indicators are possible. For example, input indicators 
relate to the resources devoted to education, output indicators to processes and 
conditions established within the education system and outcome indicators to their 
effects in educational terms. 



1L0 WHAT CHARACTERISTICS SHOULD PERFORMANCE INDICATORS 
HAVE? 

11.1 Analysis of the research literature and reviews on performance indicators for education 
suggests that they should have the following characteristics: 

* Significance: They should measure things that are valued as important and can 
be changed by the actions of teachers and school managers. 

* Validity: They should be widely accepted and understood and measure what 
they are supposed to measure in an accurate way. 

* Feasibility: It should be possible to collect necessary information and apply the 
indicators within the time, resources and expertise available and without undue 
intrusion on the work of the school. 

* Comparability: Indicators should measure things which persist over trae and 
have the same meaning in different contexts, enabling comparisons to be made 
over time, indicating trends, and across schools and regions. 

* Clarity: The relationship between the indicator and the information used to 
derive it should be straightforward and understandable. Indicators should be 
described in ways that make possible to link them with other indicators for 
purposes of analysis, explanation and prediction. 

* Objectivity: It should be possible for different users of the indicator to arrive at 
consistent judgements. 



12.0 WHO IS DEVELOPING PERFORMANCE INDICATORS ? 

12.1 Much attention has been given world-wide to the identification of performance 
indicators which would help educational policy-makers and managers to come to 
decisions and various approaches have been proposed but all concerned agree that 
indicators should not be a random collection of statistics or individual components. 

12.2 HMI in Scotland has produced and published sets of performance indicators for use in 
primary and secondary schools and in further education. Future publications will 
extend this to the early stages of education and to provision for pupils with special 
educational needs. Development of performance indicators is also well advanced in 
further education where performance indicators are considered in the HMI reports 
Measuring Up and On Target. A staff development package on Performance 
Indicators in community education is in preparation. 

12.3 There have also been initiatives in the development of performance indicators by 
several edacation authorities which have tailored the HMI indicators to their own needs 
or have produced alternative sets. Cross-referencing, where required, can help schools 
avoid confusion. 

12.4 SOED is playing a leading role in one aspect of the work of an international project co- 
ordinated by the Organisation for Economic Co-operation and Development (OECD) 
which is developing indicators across a range of aspects of education. 




-9- 



Ii 



12.5 The availability of data on many quantitative indicators has resulted in their being the 
focus of most of the international developments to date. Thus, for example, the 
indicators in Education at a Glance, (1992) the first report of the OECD project, deal 
particularly with the demography, costs, resources and outcomes of education in 
OECD countries. OECD work is continuing on other areas such as attitudes and 
expectations (the area co-ordinated by SOED), but the qualitative basis for such 
indicators makes it more difficult to establish common criteria. 



13.0 WHAT SORT OF INDICATORS HAVE BEEN PRODUCED BY HMI ? 

13.1 HMI has designed sets of performance indicators in the light of the above criteria, 
within the framework of key areas identified for school audit as outlined in the first 
section of this paper. 

13.2 These performance indicators have been based on the experience of evaluating the 
work of schools. Most are qualitative indicators of processes, conditions and standards 
of attainment within the school but quantitative indicators have been developed for 
examination results in secondary schools. 

13.3 In each of the key areas a number of contributory aspects of the work has been 
identified and a performance indicator developed for each. 

13.4 HMI performance indicators are structured firstly by specifying for each a number of 
themes to assist teachers in understanding exactly what to be taken into account. 

13.5 Secondly, for each indicator four levels of performance are identified, the first and third 
of which of which are illustrated by factors which characterise them in terms of each of 
the themes of the indicator. 

13.6 The four levels of performance are: 

* Major strengths - a very good performance 

(summarised as very good) 

* Strengths outweigh any weaknesses - some improvement desirable 

(good) 

* Strengths outweighed by weaknesses - improvement needed in 

certain aspects 
(fair) 

* Major weaknesses - an unsatisfactory performance 

(unsatisfactory) 

13.7 The illustrations are based on observation of the work of schools. They are intended to 
relate to the implementation of principles rather than to very specific "sine qua non" 
criteria. Accordingly they are not intended to be comprehensive or exclusive and they 
also serve as models for schools which might wish to develop indicators for their own 
particular priorities. The aim of publication is to assist schools in their own work rather 
than provide a prescriptive approach to educational audit. 

13.8 As stated above, HMI performance indicators are mainly qualitative indicators. 
However, following the use of them in making evaluations, it is possible to derive 
quantitative data on, for example, the proportion of cases in which performance in a 
particular aspect is unsatisfactory. This is the approach adopted in Standards and 
Quality in Scottish Schools 1991/92. 



13.9 The HMI system of indicators was piloted in school inspections and developed with the 
co-operation of a group of education authorities. These indicators form the basis for all 
HMI inspections and for Audit Unit reports such as Standards and Quality in Scottish 
Education, 1991192. Communication between schools and authorities is likely to be 
assisted by a shared understanding of the factors involved and the specification of 
levels of performance; a common language is likely to minimise misunderstandings. 



14.0 WHAT ARE THE CRITICAL PERFORMANCE INDICATORS ? 

14.1 The indicators that are used should include quantitative indicators, qualitative 
indicators based on professional consideration and also indicators based on judgements 
which take account of the views of parents and pupils. Indicators in the last category, 
Ethos Indicators, deal with aspects which are central to the well-being of a school. 

14.2 Within the HMI set of indicators, each one relates to a vital part of the life and work of 
every school and it is important that a school audit should take account of all of them. 

14.3 However, it is possible to identify some of these areas where performance indicators 
will be particularly crucial to consideration of the quality of education provided, 
including: 

quality of learning and teaching 

pupil progress and attainment 

quality cf courses (individual courses and the curriculum) 
quality of ethos (including views of parents, pupils and teachers) 
quality of management (e.g. staff, finance) 
levels of attendance and truancy 
unit costs 

leavers' destinations 



15.0 USING PERFORMANCE INDICATORS TO BEST EFFECT ? 

15.1 No single indicator should be used in isolation to make firm judgements; a question 
raised by one may be confirmed or refuted by another. An individual indicator will not 
lead to an answer but to the right kind of question. 

15.2 A small number of performance indicators is unlikely to give an adequate picture of the 
work of a school. The use of any indicator, or group of indicators, should be set in the 
broad context of the other work of the school. 

15.3 It would be wrong in any case to focus too narrowly within a school on certain aspects 
of its work to the detriment of the breadth of education which it offers. 

15.4 The impetus for accountability, quality, and value for money in education points to the 
need for the use of performance indicators as an integral part of the evaluation of 
educational provision through school audit. 




15.5 But there should be an expectation that when the indicators are applied over a period of 
time they will provide evidence of improvement. 

15.6 This will not happen merely because indicators exist, are accepted and are used. A 
school must plan for improvement, using the performance indicators within school 
audit to identify targets which can be overtaken to bring about the desired 
improvements in the quality of education provided. 



•i 
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